Most existing text-video retrieval methods focus on cross-modal matching between the visual content of offline videos and textual query sentences. However, in real scenarios, online videos are frequently accompanied by relevant text information such as titles, tags, and even subtitles, which can be utilized to match textual queries. This inspires us to generate associated captions from offline videos to help with existing text-video retrieval methods. To do so, we propose to use the zero-shot video captioner with knowledge of pre-trained web-scale models (e.g., CLIP and GPT-2) to generate captions for offline videos without any training. Given the captions, one question naturally arises: what can auxiliary captions do for text-video retrieval? In this paper, we present a novel framework Cap4Video, which makes use of captions from three aspects: i) Input data: The video and captions can form new video-caption pairs as data augmentation for training. ii) Feature interaction: We perform feature interaction between video and caption to yield enhanced video representations. iii) Output score: The Query-Caption matching branch can be complementary to the original Query-Video matching branch for text-video retrieval. We conduct thorough ablation studies to demonstrate the effectiveness of our method. Without any post-processing, our Cap4Video achieves state-of-the-art performance on MSR-VTT (51.4%), VATEX (66.6%), MSVD (51.8%), and DiDeMo (52.0%).

Vision-language models (VLMs) that are pre-trained on large-scale image-text pairs have demonstrated impressive transferability on a wide range of visual tasks. Transferring knowledge from such powerful pre-trained VLMs is emerging as a promising direction for building effective video recognition models. However, the current exploration is still limited. In our opinion, the greatest charm of pre-trained vision-language models is to build a bridge between visual and textual domains. In this paper, we present a novel framework called BIKE which utilizes the cross-modal bridge to explore bidirectional knowledge: i) We propose a Video Attribute Association mechanism which leverages the Video-to-Text knowledge to generate textual auxiliary attributes to complement video recognition. ii) We also present a Temporal Concept Spotting mechanism which uses the Text-to-Video expertise to capture temporal saliency in a parameter-free manner to yield enhanced video representation. The extensive studies on popular video datasets (ie, Kinetics-400 & 600, UCF-101, HMDB-51 and ActivityNet) show that our method achieves state-of-the-art performance in most recognition scenarios, eg, general, zero-shot, and few-shot video recognition. To the best of our knowledge, our best model achieves a state-of-the-art accuracy of 88.4% on challenging Kinetics-400 with the released CLIP pre-trained model.

Motivation: Enhancers are important cis-regulatory elements that regulate a wide range of biological functions and enhance the transcription of target genes. Although many state-of-the-art computational methods have been proposed in order to efficiently identify enhancers, learning globally contextual features is still one of the challenges for computational methods. Regarding the similarities between biological sequences and natural language sentences, the novel BERT-based language techniques have been applied to extracting complex contextual features in various computational biology tasks such as protein function/structure prediction. To speed up the research on enhancer identification, it is urgent to construct a BERT-based enhancer language model. Results: In this paper, we propose a multi-scale enhancer identification method (iEnhancer-ELM) based on enhancer language models, which treat enhancer sequences as natural language sentences that are composed of k-mer nucleotides. iEnhancer-ELM can extract contextual information of multi-scale k-mers with positions from raw enhancer sequences. Benefiting from the complementary information of k-mers in multi-scale, we ensemble four iEnhancer-ELM models for improving enhancer identification. The benchmark comparisons show that our model outperforms state-of-the-art methods. By the interpretable attention mechanism, we finds 30 biological patterns, where 40% (12/30) are verified by a widely used motif tool (STREME) and a popular dataset (JASPAR), demonstrating our model has a potential ability to reveal the biological mechanism of enhancer. Availability: The source code are available at https://github.com/chen-bioinfo/iEnhancer-ELM Contact: junjiechen@hit.edu.cn and junjie.chen.hit@gmail.com; Supplementary information: Supplementary data are available at Bioinformatics online.

强化学习（RL）和连续的非线性控制已成功部署在复杂的顺序决策任务的多个领域中。但是，鉴于学习过程的探索性质和模型不确定性的存在，由于缺乏安全保证，将它们应用于安全至关重要的控制任务是一项挑战。另一方面，尽管将控制理论方法与学习算法相结合，但在安全RL应用中显示了希望，但安全数据收集过程的样本效率尚未得到很好的解决。在本文中，我们提出了一个\ emph {可证明的}示例有效的情节安全学习框架，用于在线控制任务，以利用未知的非线性动力学系统来利用安全的探索和剥削。特别是，框架1）在随机设置中扩展控制屏障功能（CBF），以在模型学习过程中实现可证明的高概率安全性，2）整合基于乐观的探索策略，以有效地将安全探索过程与学习的动态有效地指导安全探索过程对于\ emph {接近最佳}控制性能。我们对与理论保证的最佳控制器和概率安全性的偶发性遗憾进行了正式分析。提供了仿真结果以证明所提出算法的有效性和效率。

本文介绍了一个新颖的社会偏好意识分散的安全控制框架，以解决避免多机构碰撞的责任分配问题。考虑到代理不一定会以对称方式进行合作，本文着重于具有不同合作水平的异质代理之间的半合作行为。利用社会价值取向（SVO）来量化个人自私的思想，我们提出了一个新颖的责任相关社会价值取向（R-SVO）的新颖概念，以表达成对代理之间的预期相对社会含义。这用于根据相应的责任份额来重新定义每个代理商的社会偏好或个性，以促进协调方案，例如所有代理商以不对称方式互动的半合件碰撞避免。通过通过拟议的本地成对责任权重纳入这种相对的社会影响，我们为个人代理人开发了与责任相关的控制屏障功能的安全控制框架，并通过正式可证明的安全保证可以实现多代理碰撞的避免。提供了模拟来证明在多个多代理导航任务中所提出的框架的有效性和效率，例如位置交换游戏，自动驾驶汽车公路公路坡道合并方案以及圆形交换游戏。

现有视觉语言预训练（VLP）方法主要依赖于配对的图像文本数据集，这些数据集由大量人类劳动注释，或者从互联网上爬行，然后是精心制作的数据清洁技术。为了减少对良好的图像文本对的依赖，有望直接利用仅大规模的仅文本和仅图像的语料库。本文提出了一种数据增强方法，即跨模式cutmix（CMC），用于在未配对的VLP中进行隐式跨模式对齐学习。具体而言，CMC将自然句子从文本视图转换为多模式视图，在该视图中，句子中的视觉词语单词被带有相似语义的各种图像贴片随机替换。拟议中的CMC有几个吸引人的礼节。首先，它增强了数据多样性，同时保持语义含义完好无损地解决了对齐数据稀缺的问题；其次，通过将跨模式噪声连接到单模式数据上，它指导模型以学习跨模态的令牌级相互作用，以更好地降级。此外，我们提出了一种名为VLMIXER的新的未配对VLP方法，该方法将CMC与对比度学习集成在一起，以将Uni-Mododal和多模式视图汇总在一起，以在不同模式之间进行更好的实例级别对齐。在五个下游任务上进行的广泛实验表明，VLMIXER可以超过以前最先进的未配对VLP方法。

多机器人系统（MRS）是一组协调的机器人，旨在相互合作并完成给定的任务。由于操作环境中的不确定性，该系统可能会遇到紧急情况，例如未观察到的障碍物，移动车辆和极端天气。蜂群等动物群体会引发集体紧急反应行为，例如绕过障碍和避免掠食者，类似于肌肉条件的反射，该反射组织局部肌肉以避免在第一反应中避免危害，而不会延迟通过大脑的危害。受此启发，我们开发了一种类似的集体反射机制，以使多机器人系统应对紧急情况。在这项研究中，基于动物集体行为分析和多代理增强学习（MARL），开发了一种由生物启发的紧急反应机制（MARL）开发的集体条件反射（CCR）。该算法使用物理模型来确定机器人是否经历了紧急情况。然后，通过相应的启发式奖励增强了涉及紧急情况的机器人的奖励，该奖励评估紧急情况和后果并决定当地机器人的参与。 CCR在三个典型的紧急情况下进行了验证：\ textit {湍流，强风和隐藏障碍物}。仿真结果表明，与基线方法相比，CCR以更快的反应速度和更安全的轨迹调整来提高机器人团队的紧急反应能力。

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

In this paper, a semantic communication framework for image transmission is developed. In the investigated framework, a set of servers cooperatively transmit images to a set of users utilizing semantic communication techniques. To evaluate the performance of studied semantic communication system, a multimodal metric is proposed to measure the correlation between the extracted semantic information and the original image. To meet the ISS requirement of each user, each server must jointly determine the semantic information to be transmitted and the resource blocks (RBs) used for semantic information transmission. We formulate this problem as an optimization problem aiming to minimize each server's transmission latency while reaching the ISS requirement. To solve this problem, a value decomposition based entropy-maximized multi-agent reinforcement learning (RL) is proposed, which enables servers to coordinate for training and execute RB allocation in a distributed manner to approach to a globally optimal performance with less training iterations. Compared to traditional multi-agent RL, the proposed RL improves the valuable action exploration of servers and the probability of finding a globally optimal RB allocation policy based on local observation. Simulation results show that the proposed algorithm can reduce the transmission delay by up to 16.1% compared to traditional multi-agent RL.

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.